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SUMMARY 

A radar target acquisition research study was conducted to access the 

effects of two levels of 13 radar sensor, display, and mission parameters 

on operator tactical target acquisition. A saturated fractional -factorial 

screening design was employed to examine these parameters. Data analysis 
2 

computed ETA values for main and second-order effects for the variables 
tested. Ranking of the research parameters in terms of importance to system 
design revealed four variables (radar coverage, radar resolution/multiple 
looks, display resolution, and display size) accounted for 50 percent of 
the target acqu'sition probability variance. 

1.0 INTRODUCTI ON 

Synthetic aperture radar (SAR) operator performance research was begun in 
the mid-1960's and has continued at a modest level to the present time. 

Most of this research has been directed towards defining SAR sensor charac- 
teristics for locating large, location known, fixed targets. Only recently 
have SAR sensors systems appeared suitable for operators to locate and 
designate small tactical targets. There have been several limited initial 
investigations of SAR operator performance for detection/recognition of 
small targets [1, 2, 3, 4, 5, 6]. 

The objective of this research study was to identify the important radar, 
display, and mission variables in a quick response tactical target acquisi- 
tion mission. The results of the study would identify variables which have 
a m'jor affect on operator/system performance and which variables are 
unimportant. This information would allow designers and mission planners 
to make better design decisions. 
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2.0 METHOD 

To achieve the research objective, a number of variables must be varied and 
controlled. Recent work by Simon [7] has resulted In the development of 
economical mult1fa*tor behavioral research designs. The economy Is based 
on the fact that there are very few meaningful Interactions beyond the 
second order In behavioral research. Simon recommends a general behavioral 
research strategy that proceeds from a large multi-factor screening study to 
research In which functional relationships are described using central- 
composite designs and finally to performance prediction using system simu- 
lation. The first step of this research strategy was adopted fi,r the pre- 
sent study. 

The quick response mission scenario provided the foundation for the i- 
Ing study In which two levels of 13 parameters were Investigated. Thr, u 
study parameters are summarized In Table 1. 

2 . 1 RADAR IMAGERY, TEST EQUIPMENT. REFERENCE MATERIAL AND OPERATORS 
High resolution UPD-4 synthetic aperture radar imagery recorded during 
military maneuvers In West Germany was used. The raw radar signal film 
was reprocessed to provide the two levels of radar resolution required In 
the study. 

The Imagery was screened for selection of 32 simple and complex background 
scenes In which radar tactical targets were embedded. The composite target 
scenes were mounted In glass slides and projected with a 12.7 cm optical 
projector onto a back-projection screen. The focus and projected Image 
size on the screen was adjusted to produce the required levels of display 
resolution and display size. 

Army Map Service (AMS) 1:50,000 topographic charts and mission/target 
descriptor information on 12.7- by 17.8-cm index cards served as briefing 
and reference materials. Operator/subjects used the charts during target 
briefing, and the charts were available during each test trial. 

Tactical Air Command F-111 Weapon Systems Officers (WSOs) from Nellis Air 
Force Base served as radar operators in the study. Eight WSOs were selected 
from an initial sample of 60. 
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TABLE 1 

STUDY PARAMETERS AND LEVELS 


Parameter 

Low Level 

High Le 1 

Radar Resolution/ 
Looks 

12.2 meters/4 looks 

High Re;; on/ 

1 look 

Radar Coverage 

4572 X 4572 meters 

1524 X 1514 meters 

Display Resolution 

5j0 X 500 elements 

2000 X 20 J' ele- 
ments 

Display Size 

12.7 X 12.7 cm 

25.4 X 25.4 cm 

Target Type 

C 'oy 

AAA site 

Background Com- 
plexity 

Compl ex 

Simple 

Target Difficulty 

Difficult 

Easy 

Ta-^et Intelligence 

Low 

High 

Target Reference 
Point 

No 

Yes 

N,;vigation System 
Error 

+610 meters 

+91 meters 

Navigation Error 
Cursor Design 

1 

Crosshair 

Circle 

Operator Viewing 
Distance 

71 cm 

36 cm 

Mission Time 
Available 

20 seconds 

60 seconds 


2.2 EXPERIMENTAL DESIGN 

The effects of 13 variables were examined using a fractional-factorial 

design. To completely specify the effects of 13 independent variables, each 

at two levels, and all of their interactions in a full factorial design 
1 3 

would require 2 or 8192 observations per subject. Clearly, this was an 
impracticably large number of conditions and some modification was necessary. 
In most behavioral research it is reasonable to assume that higher order 
interactions account for very little variance. With this assumption, a 
research strategy which, as a first step, examines only the main effects of 
each variable, is an economi .1 research approach. 
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Slaon [7] describes an econonical sampling strategy* referr«Hl to as a 

13 

screening design, which uses a fractional replicate of a 2 factorial 
design. The basic (tesign consists of 16 selected combinations of the 13 
Independent variables which Is augmented with an additional 16 conditions. 
The 32 total experimental conditions allow an estltoate of the main effects 
along with a number of aliased two- factor Interaction strings. 

2.3 PROCEDURES 

Each operator received a one-hour familiarization briefing, 32 training 
trials, and 32 test trials over a two-day period. Each operator was given 
a verbal briefing which Included: 1) a description of the study (^Jectlves 
and an overview jf what the operator would be doing, 2) a discussion of the 
application of SAR to the quick response mission, 3) a description of the 
radar, display, and mission parameters the operator would experience during 
tile study, 4) a review of the UTN map grid system, and 5) a series of train- 
ing trials to familiarize the operator with the parameters under Investiga- 
tion. A warm-up trial was given before each test trial because of the large 
differences In conditions that existed among the test trials due to changes 
of tile 13 study parameters from trial to trial. Target acquisition time, 
orobablllty, and operator confidence estimates were the performance measures 
recorded. 

3.0 RESULTS 

The objective of this study was to determine what parameters are Important 

determiners of operator performance in a simulated tactical strike mission. 

This screening study with 13 parameters was designed to accomplish this 

objective. In this type of study the percent of variance attributable to 

the different study parameters (ETA squared) Is the measure used to make 

2 

this determination. The ETA values for each of the 13 parameters studied 
are given for the three performance measures— target acquisition time, 
probability of correct target acquisition, and operator confidence 
judgment. 

The time measure used in the analysis was the percent of time used of the 
total mission time available. This measure was used to take incorrect tar- 
get acquisitions into account in the an* sis of the time data. If an 
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operator found the wrong target or ran out of tine, the nxinM tiae availa- 
ble. 20 or 60 seconds depending on the nission tiae available for that trial, 
was used as the time score. The ratio of tiae available to tiae taken to 
acquire the target was the value used in the analysis. 

Operator confidence data were analyzed by assigning a value of 4 for "high* 
confidence, 3 for "aediua" confidence, 2 for "low" confidence, and 1 for 
trials where the operators ran out of tiae. Although these values cannot be 

assuaed to be on an interval scale which, strictly speaking, is required by 

2 2 
the analysis for ETA , they allow an approximate estimation of ETA values 

for the operator confidence judgment data. 

Following the presentation of results for percent of variance attributable 
to the parameters studied, those parameters determined as important will be 
discussed. 

3.1 VARIANCE ATTRIBUTABLE TO STUDY PARAHETERS 

Evaluation of the ETA^ values indicates that radar coverage, display reso- 
lution, and radar resolution were the principal parameters that affected 
operator target acquisition performance. Radar coverage was the most 
important variable (smaller coverage produced better performance) accounting 
for 30.1, 22.2, and 26.6 percent ETA^ values for the time, probability, and 
confidence judgment measures. Display resolution, the second most important 
parameter, accounted for 13.2, 12.5, and 17.6 percent of variance, respec- 
tively, for the time, probability, and confidence judgment measures. Radar 

resolution was the third most important parameter for operator time and 

2 

probability performance— 9.1 and 10.5 percent ETA , respectively. Interest- 
ingly, radar resolution was not an important parameter in the operator's 
confidence judgments (0.8 percent ETA ). The operator's judgments of their 
acquisition performance were unaffected by presentation of a high resol u- 
tion/l-look or a 12.2 meter/4-1ook radar image. 

A second group of three parameters had moderate effects on operator perfor- 
mance; these parameters were display size, navigation error, and target type. 
Display size was the most important of these parameters, accounting for 4.9, 
7.0, and 3.9 percent of the variance for time, probability, and confidence 
judgment measures. Navigation error accounted for 5.6, 3.1, and 2.3 percent 
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of the variance for the time, probability* and confidence judgment measures. 
Although target type had a moderate effect on target acquisition time (5.4 
percent of the variance), it had a small effect on target acquisition pro- 
bability 0*4 percent of the variance) and no effect on the operator's con- 
fidence judgments (less than 0.1 percent of the variance). 

Four of the six remaining parameters that accounted for a small but meaning- 
ful (2 to 3) percent of the variance included: navigation error cursor 

design, target reference point, target difficjity, and mission time availa- 
ble. The two variables having no affect on operator performance were the 
amount of target intelligence and operator viewing distance. 

One second-order interaction which accounted for a relatively large percent 
of the variance was i<tentified in tiie study. This was the interaction 
between radar coverage and display resolution. Display resolution had a 
much greater effect on time and probability performance with the large radar 
coverage. At the small coverage, the 2000-line display resolution was only 
slightly superior to the 500-line display. 

In the remainder of this discussion of results, those parameters determined 
to be of major or moderate importance will be addressed separately. The 
inq>1ication of the results to SAR tactical strike systems design will be 
discussed, and recomnendations for additional research to establish quanti- 
tative functional relationships among the parameters and operator perfor- 
mance will be made. 

3.2 OPERATOR PERFORMANCE OBTAINED WITH THE STUDY PARAMETERS 
The performance estimates that will be discussed for each of the two levels 
of the parameters are averaged across the high and low levels of the other 
12 parameters. 

3.2.1 RADAR COVERAGE 

The 1524-meter coverage resulted in the best performance. The large effect 
of radar coverage in favor of the smaller coverage was a surprising result. 
Past SAR tactical target acquisition research [3] found a small effect due 
to radar coverage in favor of larger coverage. 
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Inspection of radar scenes at the two coverages indicates that scale factor 
Is probably the underlying cause. At the 4572-meter coverage, Ihe Individ- 
ual target returns that made up the linear and pattern targets are small 
and difficult to see; even the complete target made up of several radar 
returns was small at the 4572-meter coverage. With the 1524-meter coverage, 
the targets are much easier to see. The Increased performance obtained with 
the larger display size, discussed later, supports this hypothesis. For 
small targets, displayed scale factor of the SAR scenes Is an Important 
design consideration, which would argue for a small radar coverage. 

3.2.2 DISPLAY RESOLUTION 

The second most Important parameter was display resolution. The 2000-line 
display resolution was determined to be considerably better than the 500- 
line resolution display. Previous research [4] found that operator acquisi- 
tion of large targets was not affected by variation of display resolution; 
although, qualitative evaluation of tactical targets in that same study 
indicated that tactical targets might be affected by display resolution. 

In the current study, display resolution was varied by defocusing the opti- 
cal projection system and adding TV line structure to the radar images. 
Therefore, two factors were varied in the two display resolutions— 
resolution and line structure. The interaction between display resolution 
and radar coverage indicate? that line structure was probably the more 
important factor. At 1524-meter coverage, there was only a small perfor- 
mance advantage for the 2000-line resolution display over the 500-line 
display, while at the 5472-meter coverage performance was significantly 
better with the 2000-line resolution display. 

The hypothesis is that the coarse line structure of the 500-line resolution 
display interferred with the target patterns at the 4572-meter radar cov- 
erage, because the line structure was large relative to the size of the 
target returns. The line structure of the 2000-line resolution display was 
small relative to the target patterns at the 4572-meter coverage and hence 
did not interfere with target patterns. At 1524-meter coverage, the line 
structure of the 500- line resolution display did not interfere with the 
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target return patterns, and performance was only slightly poorer compared 
to the 2000-line resolution display. 

3.2.3 RESOLUTION/HULTIPLE LOOKS 

As expected, resol utirni/multi pie looks had a major effect on operator target 
acquisition. The high resolution/1 -look condition resulted in substantially 
improved performance over the 12.2-meter resol ution/4- look condition. Fu- 
ture research should be conducted to establish functional relationships 
among radar resolution, radar multiple looks, and operator target acquisi- 
tion performance for use by radar sysV: s designers. The high resolution/ 
l-1ook and 1 2. 2-me ter/4- look conditions represent reasonable upper and 
bounds for such research. 

3.2 4 NAVIGATION SYSTEM ERROR 

The +91-meter and jjSlO-meter navigation errors investigated had a moderate 
affect on operator performance. The smaller navigation error, as one would 
expect, resulted in more rapid operator target acquisition and higher pro- 
bability of correct target acquisition. 

3.2.5 DISPLAY SIZE 

All other things being equal, a large display size allows SAR video tc be 
displayed at a larger scale factor than a small display size. As discussed 
previously, displayed SAR video scale factor may be an important factor in 
SAR operator tactical target acquisition. The 25.4-cm (larger scale factor) 
display resulted in better performance than did the 12.7-cm display. 

Display size coupled with display resolution, radar coverage, radar resolu- 
tion, and operator viewing distance determine the operator's ability to 
extract displayed sensor information. In this screening study, it was not 
possible to determine the potentially complex interactions among these 
parameters. Since the scale factor of the displayed SAR video appears to 
be an important factor in SAR operator target acquisition performance and 
since cockpit real estate is precious in attack aircraft, it is important 
that future research ba conducted to determine the relationships among dis- 
play size, radar coverage, radar resolution, and operator viewing distance. 
If aircraft cockpit considerations dictate a small display, other design 
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options may have to be exercised to avoid unnecessarily limiting the opera- 
tor's performance potential. The suggested research would provide the data 
to make intelligent design decisions. 

3.2.6 TARGET TYPE 

Target/background characteristics are known to be major sources of variance 
in ground mapping sensor target acquisition studies. Target type, back- 
ground •‘inplexity. and target difficulty were controlled variables in this 
study, primarily to extract and measure this source of variance. Of these 
three variables, target type caused the most performance variation. The 
extended targets (convoys) were acquired faster and more often than were the 
pattern targets (AAAs). Post hoc analysis of the radar images leads us to 
conclude that scene clutter and raster structure tended to interfere more 
with the perception of the pattern targets than the linear targets, hence 
the poorer performance obtained with pattern targets. 

4.0 CONCLUSION 

This research was perfonned to establish the relative importance among radar 
sensor, display, and mission variables on ope»"ator tactical target acquisi- 
tion performance for application to the quick response mission. The results 
of the research provided the data necessary to identify which were the 
important variables as a first step towards defining a viable system and 
future research requirements. 

ACKNOWLEDGEMENTS 

Contractual support for the study was provided by the Air Force Avionics 
Laboratory under AF Contract F33615-76-C1209. Mr. F. P. Johnson was the 
Air Force contract monitor. The authors would like to extend their appre- 
ciation to Lt. Colonel K. H. Kenworthy an:I Major R. Monsell of the Air 
Force Fighter Weapons Center for their expert critique of the study pro- 
cedures and the scheduling of TAC operators. 


III-7-9 



REFERENCES 


1. Hershberger, M. L. and Craig, D. W. Synthetic array radar utilization 

and display . Tech. Rpt. AFAL-TR-69-367. Wright-Patterson Air 
Force Base, Ohio: Air Force Avionics Laboratory, April 1970. 
(Secret) 

2. Hershberger, M. L. and ''raig, 0. W. A preliminary simulation of an 

advanced weapon delivery system for low altitude bombing . 

Tech. Mem. TM72-03. Culver City, California: Hughes Aircraft 

Company, Display Systems Laboratory, February 1972. 

3. Humes, J. M., Craig, D. W., Poplawski, T. C., Guerin, D. F. and 

Hershberger, M. L. Synthetic array radar operator perfor- 
mance simulation studies . Tech. Rpt. AFAL-TR-74-63. Wright- 
Patterson Air Force Base, Ohio: Air Force Avionics Labora- 
tory, August 1974. (Confidential) 

4. Humes, J. M., Craig, D. W., Poplawski, T. C. and Hershberger, M. L. 

Synthetic array radar operator performance evaluation studies . 
Tech. Rpt. AFAL-TR-74-158. Wright-Patterson Air Force Base, 
Ohio: Air Force Avionics Laboratory, Novetrt)er 1974. 

(Confidential ) 

5. Lamonica, 6. L. The effects of image quality on synthetic aperture 

radar operator performance . Tech. Rpt. AFAL-TR-74-333 Volume 
II. Wright-Patterson Air Force Base, Ohio: Air Force 
Avionics Laboratory, March 1975. (Confidential) 

6. Hershberger, M. L., Scanlan, L. A., and Craig, D. W. Crew size evalua- 

tion for all weather strike aircraft . Tech. Rpt. AFAL-TR-76- 
79. Wright-Patterson Air Force Base, Ohio: Air Force Avion- 
ics Laboratory, June 1976. 

7. Simon, C. W. Economical multi factor designs for human factors engineer- 

ing , Tech. Rpt. P73-326. Culver City, California: Hughes 

Aircraft Company, Display Systems Laboratory, June 1973. 


III-7-10 



